Blur-kernel estimation from spectral irregularities

ABSTRACT

A method for deriving a blur kernel from a blurred image is provided herein. The method may include the following steps: obtaining a blurred image B, being a product of a blur kernel k applied to an original image I; calculating f θ (x)=R d*Pθ(B) (x) for every angle θ, wherein R denotes an autocorrelation operator, P θ  denotes a projection operator of based on angle θ, and d denotes a one dimensional differentiation filter; estimating spectral power of the blur kernel based on a given support parameter; estimating the blur kernel k using a phase retrieval algorithm, based on the estimated spectral power of the blur kernel; updating the support parameters; and repeating the estimating of the spectral power, the estimating of the kernel and the updating of the support parameters in an iterative, to yield the blur kernel.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Non Provisional application claiming the benefitof U.S. Provisional Patent Application No. 61/663,747, filed on Jun. 25,2012, which is incorporated herein in its entirety.

FIELD OF THE INVENTION

The present invention relates generally to the field of imageenhancement, and in particular, to reducing blur based on spectralirregularities estimation.

BACKGROUND OF THE INVENTION

In many practical scenarios, such as hand-held cameras or ones mountedon a moving vehicle, it is difficult to eliminate camera shake. Sensormovement during exposure leads to unwanted blur in the acquired image.Under the assumption of white noise and spatially-invariant blur acrossthe sensor this process may be modeled by Eq. (1) below:B(x)=(I*k)(x)+η(x)  (1)where * denotes the convolution operation, B is the acquired blurryimage, k is the unknown blur kernel and η(x) is a zero-mean,identically- and independently-distributed noise term at every pixelx=(x,y). Blind image deconvolution, the task of removing the blur whenthe camera motion is unknown, is a mathematically ill-posed problemsince the observed image B does not provide enough constraints fordetermining both I and k. Most deblurring techniques, therefore,introduce additional constraints over I and k. The most common frameworkfor incorporating such prior knowledge is through maximum a posteriori(MAP) estimation. Norms favoring sparse derivatives are often used todescribe I as a natural image. While not being failure-free, thisapproach was shown to recover very complex blur kernels and achieveimpressive deblurred results. However, the maximization of theseestimators is a time consuming task involving the computation of thelatent image multiple times.

An alternative approach to the problem, which did not receive as muchattention, extracts the blur kernel k directly from the blurry image Bwithout computing I in the process. The basic idea is to recover k fromthe anomalies that B shows with respect to the canonical behavior ofnatural images. One solution known in the art is to compute the 1Dautocorrelation of the derivative of B along the sensor movementdirection. Normally, image derivatives are weakly correlated and hencethis function should be close to a delta function. The deviation fromthis function provides an estimate for the power spectrum (PS) of thekernel, |{circumflex over (k)}(ω)|².

One known approach is to recover the PS of two-dimensional kernels anduse the eight-point Laplacian for whitening the image spectrum. The blurkernel is then computed using a phase retrieval technique that estimatesthe phase by imposing spatial non-negativity and compactness. Thisapproach consists of evaluating basic statistics from the input B and,unlike methods that use MAP estimation, does not involve repeatedreconstructions of I. While this makes it favorable in terms ofcomputational-cost, the true potential of this approach in terms ofaccuracy was not fully explored.

Current solutions were directed at the removal of image blur due tocamera motion. Blind-deconvolution methods that recover the blur kernelk and the sharp image I rely on various regularities natural imagesexhibit. The most-dominant approach for tackling this problem, in thecontext of spatially-uniform blur kernel, is to formulate and solve aMAP problem. This requires the minimization of a log-likelihood termthat accounts for Eq. (1) plus additional prior terms that score theresulting image I and kernel k. One solution uses an autoregressiveGaussian prior for I(x) and another solution uses a similar Gaussianprior over high-frequencies (derivatives) of I. Both priors are blind tothe phase content of k and are not sufficient for recovering it. Anotherapproach further assumes that the blur is symmetric (zero phase) whileanother approach incorporates adaptive spatial weighting which breaksthis symmetry. Yet in another known approach, the Gaussian image prioris replaced with a Laplace distribution defined by the l1 norm over theimage derivatives. This choice is more consistent with the heavy-tailedderivative distribution observed in natural images. Yet anothersuggestion of the current art shows that this prior is not sufficientfor uniqueness and may result in degenerate delta kernels. Indeed,methods that rely on sparse norms often introduce additional constraintssuch as smoothness of the blur-kernel and two motion-blurred images oruse alternative image priors such as spatially-varying priors and onesthat marginalize over all possible sharp images I(x).

SUMMARY OF THE INVENTION

The present invention, in embodiments thereof, provides a method forrecovering the blur from irregularities in the statistics ofmotion-blurred images. The power-law model is derived and used todescribe the PS of natural images, which refines the traditional ∥ω∥⁻²law. This new model better accounts for biases arising from the presenceof large and strong edges in natural images. This model is used, as wellas a more accurate spectral whitening formula, to recover the PS of theblur kernel, ∥{circumflex over (k)}(ω)|², in a more robust and accuratemanner compared to previous methods following this approach. We alsodescribe several modifications to standard phase retrieval algorithms,allowing them to better converge and resolve ambiguities.

Unlike methods that rely on the presence and the identification ofwell-separated edges in the image, the purely statistical approach isaccordance with embodiments of the present invention copes well withimages containing under-resolved texture and foliage clutter, which areabundant in outdoor scenes. Similarly to known solutions the latentimage is not reconstructed repeatedly the input image is being accessedonly once to extract a small set of statistics. Thus, the core of thetechnique in accordance with embodiments of the present inventiondepends only on the blur kernel size and does not scale with the imagedimensions.

Advantageously, the method in accordance with some embodiments of thepresent invention is capable of achieving highly-accurate results, invarious scenarios that challenge other approaches, and compares wellwith the state-of-the-art. The CPU implementation of embodiments of thepresent invention achieves these results at running-times considerablyfaster than MAP-based methods.

These additional, and/or other aspects and/or advantages of the presentinvention are set forth in the detailed description which follows.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention and in order to show how itmay be implemented, references are made, purely by way of example, tothe accompanying drawings in which like numerals designate correspondingelements or sections. In the accompanying drawings:

FIG. 1 is a high level block diagram illustrating a system according tosome embodiments of the present invention;

FIG. 2 is a high level flowchart diagram illustrating a method accordingto embodiments of the present invention;

FIG. 3 shows diagrams illustrating an aspect according to embodiments ofthe present invention;

FIG. 4 shows photos illustrating an aspect according to some embodimentsof the present invention;

FIG. 5 shows photos illustrating another aspect according to someembodiments of the present invention; and

FIG. 6 shows photos illustrating yet another aspect according to someembodiments of the present invention.

The drawings together with the following detailed description make theembodiments of the invention apparent to those skilled in the art.

DETAILED DESCRIPTION OF THE INVENTION

With specific reference now to the drawings in detail, it is stressedthat the particulars shown are for the purpose of example and solely fordiscussing the preferred embodiments of the present invention, and arepresented in the cause of providing what is believed to be the mostuseful and readily understood description of the principles andconceptual aspects of the invention. In this regard, no attempt is madeto show structural details of the invention in more detail than isnecessary for a fundamental understanding of the invention. Thedescription taken with the drawings makes apparent to those skilled inthe art how the several forms of the invention may be embodied inpractice.

Before explaining the embodiments of the invention in detail, it is tobe understood that the invention is not limited in its application tothe details of construction and the arrangement of the components setforth in the following descriptions or illustrated in the drawings. Theinvention is applicable to other embodiments and may be practiced orcarried out in various ways. Also, it is to be understood that thephraseology and terminology employed herein is for the purpose ofdescription and should not be regarded as limiting.

FIG. 1 is a high level block diagram illustrating a system 100 accordingto some embodiments of the present invention. System 100 may include acomputer memory 110 configured to obtain a blurred image B 10, being aproduct of a blur kernel k applied to an original image I, wherein B andI are matrices representing pixel image arrays and k is a kernel of amatrix. System 100 may further include a computer processor 130configured to: calculate f_(θ)(x)=R_(d*Pθ(B))(x) for every angle θ,wherein R denotes an autocorrelation operator, P_(θ) denotes aprojection module 120 configured to project a two dimensional signalinto one dimension based on angle θ, and d denotes a one dimensionaldifferentiation filter 140 applied to a product of the projectionoperator P_(θ) and the blurred image B. System 100 may further includesupport parameters s_(θ) 152 set initially to argmin_(x) f_(θ)(x), and aspectral estimator 150 configured to estimate |{circumflex over (k)}|²denoting a spectral power of the blur kernel based on a given supportparameter. System 100 may further include a blur kernel estimator 160configured to estimate the blur kernel k 162 using a phase retrievalalgorithm, based on the estimated spectral power of the blur kernel|{circumflex over (k)}|². System 100 may further include an updatemodule 154 configured to update the support parameters s_(θ) toargmax_(x) (R_(Pθ(k))(x)>a·max(R_(Pθ(k)))), wherein a is constantnumber. Computer processor 130 is further configured to repeat theestimating of the spectral power |{circumflex over (k)}|², theestimating of the kernel and the updating of the support parameterss_(θ) in an expectation maximization (EM) procedure, to yield the blurkernel k.

FIG. 2 is a high level flowchart diagram illustrating a method 200according to embodiments of the present invention. It is understood thatmethod 200 is not necessarily limited to the aforementioned architectureof computer network 100 but may be carried out by similar networks.Method 200 includes the following stages: obtaining a blurred image B,being a product of a blur kernel k applied to an original image I,wherein B and I are matrices representing pixel image arrays and k is akernel of a matrix 210; calculating f_(θ)(x)=R_(d*Pθ(B))(x) for everyangle θ, wherein R denotes an autocorrelation operator, P_(θ) denotes aprojection operator of a two dimensional signal into one dimension basedon angle θ, and d denotes a one dimensional differentiation filterapplied to a product of the projection operator P_(θ) and the blurredimage B 220; setting support parameters s_(θ) to argmin_(x) f_(θ)(x)230; estimating |{circumflex over (k)}|² denoting a spectral power ofthe blur kernel based on a given support parameter 240; estimating theblur kernel k using a phase retrieval algorithm, based on the estimatedspectral power of the blur kernel |{circumflex over (k)}|² 250; updatingthe support parameters s_(θ) to argmax_(x)(R_(Pθ(k))(x)>a·max(R_(Pθ(k)))), wherein a is constant number 260; andrepeating the estimating of the spectral power |{circumflex over (k)}|²,the estimating of the kernel and the updating of the support parameterss_(θ) in an expectation maximization (EM) procedure, to yield the blurkernel k 270.

Going now into more details of the embodiments of the present invention,it has been already pointed out that the following power-law describesthe power spectra of images of natural scenes as in Eq. (2) below:|Î(ω)|²∝∥ω∥^(−β)  (2)

Where I is a natural image, Î its Fourier transform and ω denotes thefrequency coordinates. Compact second-order derivative filters, forexample l_(x)=[−1, 2,−1], provide a good approximation for |ω_(x)|²especially at low frequencies since as shown in Eq. (3) below:

$\begin{matrix}{{{\hat{l}}_{x}( \omega_{x} )} = {{2( {{\cos( {\omega_{x}\pi} )} - 1} )} \approx {{\pi^{2}\omega_{x}^{2}} + {\mathcal{O}( \omega_{x}^{4} )}}}} & (3)\end{matrix}$where the last equality is based on Taylor approximation of cosinearound ω_(x)=0. d is denoted the as ‘square-root’ filter of l which isdefined as the symmetric filter giving l= d*d where the bar sign denotesmirroring. This factorization exists for every symmetric filter withnon-negative frequency response, i.e., Î(ω)≧0

Filtering an image obeying Eq. (2) with d results in a signal withwhitened spectrum, since as in Eq. (4) below:|

(ω)|² =|I(ω)|² ·|d(ω)² =|Î ² ·{circumflex over(l)}(ω)≈π²∥ω∥⁻²∥ω∥²=const.  (4)where the equality |{circumflex over (d)}(ω)|=Î(ω)| is the Fourieranalog of l= d*d. Thus, in the case of a blurry image B=I*k, thisprocedure can be used to obtain the following estimate of theblur-kernel PS, as in Eq. 5 below:|

(ω)|² =|Î(ω)|² ·{circumflex over (l)}(ω)·|{circumflex over (k)}(ω)|²≈c|{circumflex over (k)}(ω)|²  (5)

The Wiener-Khinchin theorem relates the PS of any signal J to itsautocorrelation by Eq. (6) below:{circumflex over (R)} _(J)(ω)=|Ĵ(ω)|²  (6)

This identity introduces real-space counterparts for the spectrumwhitening in Eq. (4) and the blur approximation in Eq. (5), that aregiven by Eq. (7) below:R _(I*d)(x)≈cδ(x), and R _(B*d)(x)≈cR _(k)(x)  (7)where δ is the Dirac's delta-function, i.e., δ (0, 0)=1 and zerootherwise.

Some teachings of the prior art exploit this regularity in naturalimages and recover the power spectra of one-dimensional blur kernels bydifferentiating the image along the blur direction (which they alsoestimate). In order to obtain the complete kernel k, they estimate itsphase using the Hilbert transform under the assumption of minimal-phaseblur. Some teachings known in the art use this regularity to recovergeneral two-dimensional kernels by whitening the image spectrum usingthe eight-point Laplacian filter. Given the estimated spectral power ofthe kernel, they compute the kernel by recovering its phase using theerror-reduction phase-retrieval algorithm.

While Eq. (2) models well certain images, the presence of long edges,both in natural and man-made scenes, undermines its accuracy. Sometimes,the power spectra show increased magnitudes along radial streaks whichare orthogonal to the strong image edges. These streaks break therotational symmetry predicted by Eq. (2). Analogously, theautocorrelation functions of images, after being whitened with d, or 1,differ considerably from the delta function, predicted in Eq. (7), alongthe directions of the strong edges. Clearly, such deviations from thepower-law in Eq. (2) undermine the accuracy of the recovered spectralpower of the kernel from Eq. (5), or equivalently R_(k) recovered fromEq. (7).

The question is, therefore, how does this discrepancy behave. To answerthis, cross-sections of the PS may be plotted along the most extremeorientations. The log-log plots of these 1D functions, can beapproximately described by an additive offset which means that the PS ofa natural image vary by multiplicative factors along differentdirections, as in Eq. (8) below:|Î(ω)|² ≈c _(θ(ω))·∥ω∥⁻²  (8)

FIG. 3 illustrates a comparison between different spectral whiteningprocedures on an image containing strong horizontal edges. In FIG. 3 thekernel PS is shown first as the try kernel PS 310 and then it isestimated by whitening the image with d 320 and 330 and l 340 and usingthe method which relies on the refined power-law in Eq. (8) 350.

Following is the Fourier slice theorem which will become instrumentalfor implementing the method according to some embodiments of the presentinvention and is given by Eq. (9) below:(

) (ω)=Ĵ(ωr _(θ))  (9)where P_(θ)(J) is a projection of a 2D signal into 1D by integrating italong the direction orthogonal to θ and r_(θ) is a unit vector in 2Dwith the orientation of θ.

Thus, ωr_(θ) parameterizes with orientation of θ using the scalar ω.When applying this theorem in out context, where J=|Î|², Eq. is obtainedas depicted below:

(ω)=|

)(ω)|² =|Î(ωr _(θ))|² ≈c _(θ)·|ω|⁻²  (10)where the first equality follows from the Wiener-Khinchin theorem whichapplies between the 1D autocorrelation of the projected image P_(θ)(I)and its power spectrum in 1D Fourier space. The last equality followsfrom Eq. (8) where the restriction to a single slice leaves only asingle unknown c_(θ).

Given a blurry image B=I*k, the relation in Eq. (10) allows us torecover the spectral power of the kernel up to a single scalar c_(θ) asin Eq. (11) below:(

)(ω)={circumflex over (l)}(ω)·|{circumflex over (B)}(ωr _(θ))|²={circumflex over (l)}(ω)·|Î(ωr _(θ))|² ·|{circumflex over (k)}(ωr_(θ))|² ≈c _(θ) ·|{circumflex over (k)}(ωr _(θ))|²  (11)resulting 1D Laplacian l= d*d. This can be written in real-space, onceagain using the Wiener-Khinchin theorem, in Eq. (12) below:(R _(d*Pθ(B)))(x)≈c _(θ) ·R _(Pθ(k))(x)  (12)

Thus, as depicted in Algorithm 1 table below the first step of thealgorithm in accordance with embodiments of the present invention is tocompute f_(θ)(x) for every angle θ. Since it is required to recover thespectral power of the kernel on a grid of pixels, the picked angles arethose that result in slices passing exactly through each pixel. Weimplemented the projection operator P_(θ) using a nearest-neighborsampling which achieved higher accuracy and runs in less time comparedto other interpolation formulae we tested.

The resulting projected values correspond to averaging of a large numberof pixels, proportional to the image dimensions n. The averaging of theindependent noise terms in Eq. (1) results in a noise reduction by afactor that renders the noise present as negligible.

Estimating spectral power of the kernel slice-by-slice according to Eq.(12) introduces a set of unknowns c_(θ), some of which are mutual tomore than one value in estimated kernel power. Additionally, the mean ofthe projected slices P_(θ)(B) is lost due to the differentiation with din Eq. (11)), or equivalently {circumflex over (k)}(0r_(θ). The missingmean values are denoted by m_(θ). c_(θ) and m_(θ) may be recovered basedon the following three kernel modeling assumptions:

1. Camera blur kernels are proportional to time period in which light isintegrated at each camera offset as it move during exposure. Thus, thesenumbers are all non-negative, i.e., k_(—)0, and so is its projectionP_(θ)(k) and the 1D autocorrelation function RP_(θ)(k);

2. Assuming a finite camera motion, the blur kernel support must bebounded; Similarly to the positivity case, the finite support is alsoinherited by P_(k) and RP_(k). Thus, for each_we have s_such that forevery |x|_s_there is RP_(k)(x)=0; and

3. Assuming that camera blur does not affect the total amount of lightreaching the sensor is expressed by R k(x)dx=1.

By repeating this procedure for all the θ an approximation for the full2D blur-kernel PS function is obtained. This approximation is used torecover the blur kernel k using a phase-retrieval algorithm that will bedescribed hereinafter.

Embodiments of the present invention provide an algorithm consisting ofan iterative EM-like procedure where there is a switch betweenestimating the kernel k(x) given the support variables sθ and estimatingsθ from the recovered k(x). There is a procedure with the initial guesssθ=argmin x f_(x).

Then, given the retrieved kernel the values are updated as seen in thetable below of Algorithm 1

  Algorithm (1)   Algorithm 1: Iterative kernel recovery. Input: blurryimage B; calculate f_(θ) = R_(d* P) _(θ) _((B)); set s_(θ) = argmin_(x)f_(θ)(x); for i = 1..N_(outer) do | estimate |{circumflex over (k)}|²given s_(θ); | estimate kernel using phase retrieval Alg. 2 given|{circumflex over (k)}|²; | update s_(θ) = argmax_(x) (R_(P) _(θ)_((k))(x) > 0.1 · max(R_(P) _(θ) _((k)))); end Output: recovered blurkernel k:

Recovering the kernel k, given its power spectrum requires estimatingthe phase component of {circumflex over (k)}(ω). There are quite a fewalgorithms that retrieve a signal or its phase given its power spectrumand additional constraints, such the signal being positive and oflimited spatial support. Largely speaking, these algorithms iterativelyswitch between Fourier and real-space domains to enforce the input PSand spatial constraints respectively.

While this algorithm performs well on an accurate input, it furtheralternates between the error-reduction and another method called hybridinput-output that avoids this problem.

Following is an exemplary algorithm that may be implemented in order tocarry out the required phase retrieval:

Algorithm (2) Algorithm 2: Phase retrieval algorithm. Input ⁢ : ⁢ ⁢ kernel ⁢⁢magnitude ⁢ ⁢ spectrum , p ⁡ ( ω ) , kernel ⁢ ⁢ size ⁢ ⁢ s ; for ⁢ ⁢ n = 1 ⁢ . . ⁢N guesses ⁢ ⁢ do | // ⁢ initiate ⁢ ⁢ the ⁢ ⁢ phase ⁢ ⁢ ϕ ⁡ ( ω ) ⁢ ⁢ randomly ⁢ ;sample ⁢ ⁢ ϕ ⁡ ( ω ) ⁢ ⁢ uniformly ⁢ ⁢ from ⁢ [ - π , π ] ⁢ : // ⁢ switch ⁢ ⁢ to ⁢ ⁢real ⁢ - ⁢ space ⁢ ⁢ using ⁢ ⁢ inv . ⁢ Fourier ⁢ ⁢ trans . ; g = - 1 ⁢ ( p · ⅇ i ⁢ ⁢ϕ ) ⁢ ; for ⁢ ⁢ m = 1 ⁢ . . ⁢ N inner ⁢ ⁢ do | // ⁢ apply ⁢ ⁢ Fourier ⁢ ⁢ domain ⁢ ⁢constraints ⁢ ; g 2 = - 1 ⁢ ( ( α ⁢ ⁢ p + ( 1 - α ) | g ^ | ) · ⅇ i · Phase ⁡( g ^ ) ) ⁢ ; // ⁢ apply ⁢ ⁢ space ⁢ ⁢ domain ⁢ ⁢ constraints ⁢ ; Ω = { x ⁢ ⁢ : ⁢ ⁢ g2 ⁡ ( x ) < 0 } ⋃ { x ⁢ ⁢ : ⁢ ⁢ x ∈ [ 0 , s ] × [ 0 , s ] } ⁢ ; define ⁢ ⁢ Δ ⁡ (x ) = { g ⁡ ( x ) if ⁢ ⁢ x ∈ Ω 0 if ⁢ ⁢ x ∉ Ω ; g ⁡ ( x ) = g 2 ⁡ ( x ) ⁢ ⁢ for ⁢ ⁢x ∉ Ω ⁢ ; if ⁢ ⁢ hybrid ⁢ ⁢ input ⁢ - ⁢ output ⁢ ⁢ step ⁢ ⁢ ( 250 ⁢ ⁢ iterations ) ⁢ ⁢then | ⁢ g ⁡ ( x ) = g ⁡ ( x ) - βΔ ⁡ ( x ) , for ⁢ ⁢ x ∈ Ω ⁢ ; else ⁢ // ⁢ error⁢ - ⁢ reduction ⁢ ⁢ step ⁢ ⁢ ( 50 ⁢ ⁢ iterations ) | ⁢ g ⁡ ( x ) = g 2 ⁡ ( x ) - γΔ⁡( x ) , for ⁢ ⁢ x ∈ Ω ⁢ ; end // ⁢ keep ⁢ ⁢ intermediate ⁢ ⁢ kernels ⁢ ⁢ obeying ⁢ ⁢spatial ⁢ ⁢ constraints ⁢ ; k m ′ = g 2 - Δ ⁢ ; end k n ⁡ ( x ) = median ⁢ { km ′ ⁡ ( x ) ⁢ ⁢ : ⁢ ⁢ N inner - 100 ≤ m ≤ N inner } ; end Output ⁢ : ⁢ ⁢ k n ⁢ ⁢with ⁢ ⁢ highest ⁢ ⁢ likelihood ⁢ ⁢ of ⁢ [ 16 ] ⁢ ⁢ when ⁢ ⁢ decoavolving ⁢ ⁢ a ⁢ ⁢small ⁢ ⁢ window ⁢ ;

FIG. 4 shows photos illustrating an aspect according to embodiments ofthe present invention. Image 410 illustrates a blurred image. Image 420is an image treated with error reduction whereas image 430 was treatedwith the phase retrieval algorithm of the aforementioned Algorithm 2. Itis understood that other phase retrieval algorithms may be used herein.

Rather than taking the kernel of the last iteration, it is suggested byembodiments of the present invention to take the pixel-wise median ofthe kernels computed, for example, in the last 250 iterations. Thekernels recovered from their PS are defined up to a translation andmirroring in space. Thus, before we compute the median we normalizetheir offset and flip them (if needed) based on their first moments.

FIGS. 4, 5, and 6 show photos illustrating an aspect according to someembodiments of the present invention. While images 410, 420, 430, 510A,520A, 530A, 510B, 520B, 530B, illustrate blurred images that weretreated by prior art methods, images 440, 540A, 540B, and 640 illustratethe original image and 450, 550A, 550B, and 650 illustrate blurredimages treated by the embodiments of the present invention. The improvedaccuracy is apparent for 450, 550A, 550B, and 650 when compared withprior art images bot with man-made landscape and natural environmentalike.

Advantageously, the method in accordance with embodiments of the presentinvention enable recovering the blur kernel in motion-blurred naturalimages based on the statistical deviations they exhibit in theirspectrum. The proposed method extracts a set of statistics from theinput image, after properly whitening its spectrum, and uses them torecover the blur. Thus, it achieves favorable running-times compared toother methods that perform MAP estimation and recover the latent imagerepeatedly in the course of estimating the kernel.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or an apparatus.Accordingly, aspects of the present invention may take the form of anentirely hardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.”

The aforementioned flowchart and block diagrams illustrate thearchitecture, functionality, and operation of possible implementationsof systems and methods according to various embodiments of the presentinvention. In this regard, each block in the flowchart or block diagramsmay represent a module, segment, or portion of code, which comprises oneor more executable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

In the above description, an embodiment is an example or implementationof the inventions. The various appearances of “one embodiment,” “anembodiment” or “some embodiments” do not necessarily all refer to thesame embodiments.

Although various features of the invention may be described in thecontext of a single embodiment, the features may also be providedseparately or in any suitable combination. Conversely, although theinvention may be described herein in the context of separate embodimentsfor clarity, the invention may also be implemented in a singleembodiment.

Reference in the specification to “some embodiments”, “an embodiment”,“one embodiment” or “other embodiments” means that a particular feature,structure, or characteristic described in connection with theembodiments is included in at least some embodiments, but notnecessarily all embodiments, of the inventions.

It is to be understood that the phraseology and terminology employedherein is not to be construed as limiting and are for descriptivepurpose only.

The principles and uses of the teachings of the present invention may bebetter understood with reference to the accompanying description,figures and examples.

It is to be understood that the details set forth herein do not construea limitation to an application of the invention.

Furthermore, it is to be understood that the invention can be carriedout or practiced in various ways and that the invention can beimplemented in embodiments other than the ones outlined in thedescription above.

It is to be understood that the terms “including”, “comprising”,“consisting” and grammatical variants thereof do not preclude theaddition of one or more components, features, steps, or integers orgroups thereof and that the terms are to be construed as specifyingcomponents, features, steps or integers.

If the specification or claims refer to “an additional” element, thatdoes not preclude there being more than one of the additional element.

It is to be understood that where the claims or specification refer to“a” or “an” element, such reference is not be construed that there isonly one of that element.

It is to be understood that where the specification states that acomponent, feature, structure, or characteristic “may”, “might”, “can”or “could” be included, that particular component, feature, structure,or characteristic is not required to be included.

Where applicable, although state diagrams, flow diagrams or both may beused to describe embodiments, the invention is not limited to thosediagrams or to the corresponding descriptions. For example, flow neednot move through each illustrated box or state, or in exactly the sameorder as illustrated and described.

Methods of the present invention may be implemented by performing orcompleting manually, automatically, or a combination thereof, selectedsteps or tasks.

The term “method” may refer to manners, means, techniques and proceduresfor accomplishing a given task including, but not limited to, thosemanners, means, techniques and procedures either known to, or readilydeveloped from known manners, means, techniques and procedures bypractitioners of the art to which the invention belongs.

The descriptions, examples, methods and materials presented in theclaims and the specification are not to be construed as limiting butrather as illustrative only.

Meanings of technical and scientific terms used herein are to becommonly understood as by one of ordinary skill in the art to which theinvention belongs, unless otherwise defined.

The present invention may be implemented in the testing or practice withmethods and materials equivalent or similar to those described herein.

While the invention has been described with respect to a limited numberof embodiments, these should not be construed as limitations on thescope of the invention, but rather as exemplifications of some of thepreferred embodiments. Other possible variations, modifications, andapplications are also within the scope of the invention. Accordingly,the scope of the invention should not be limited by what has thus farbeen described, but by the appended claims and their legal equivalents.

The inveniton claimed is:
 1. A method comprising: obtaining a blurredimage B, being a product of a blur kernel k applied to an original imageI, wherein B and I are matrices representing pixel image arrays and k isa kernel of a matrix; calculating f_(θ)(x)=R_(d*Pθ(B))(x) for everyangle θ, wherein R denotes an autocorrelation operator, P_(θ) denotes aprojection operator of a two dimensional signal into one dimension basedon angle θ, and d denotes a one dimensional differentiation filterapplied to a product of the projection operator P_(θ) and the blurredimage B; setting support parameters s_(θ) to argmin_(x) f_(θ)(x);estimating |{circumflex over (k)}|² denoting a spectral power of theblur kernel based on a given support parameter; estimating the blurkernel k using a phase retrieval algorithm, based on the estimatedspectral power of the blur kernel |{circumflex over (k)}|²; updating thesupport parameters s_(θ) to argmax_(x) (R_(Pθ(k))(x)>a·max(R_(Pθ(k)))),wherein a is constant number; and repeating the estimating of thespectral power |{circumflex over (k)}|², the estimating of the kerneland the updating of the support parameters s_(θ) in an expectationmaximization (EM) procedure, to yield the blur kernel k.
 2. The methodaccording to claim 1, wherein the projection operator P_(θ) is achievedby integrating the two dimensional signal along a direction orthogonalto the angle θ.
 3. The method according to claim 1, wherein blur kernelk represents the blur operation which transforms I to B due to movementsof a capturing device upon capturing the blurred image.
 4. The methodaccording to claim 1, wherein I is a natural image captured by an imagecapturing device.
 5. The method according to claim 1, wherein the anglesθ are selected so that each selected angle corresponds with a pixel ofthe pixel array.
 6. The method according to claim 1, further comprisingusing the retrieved blur kernel k to retrieve the original image.
 7. Themethod according to claim 1, wherein a is between 0 and
 1. 8. A systemcomprising: A computer memory configured to obtain a blurred image B,being a product of a blur kernel k applied to an original image I,wherein B and I are matrices representing pixel image arrays and k is akernel of a matrix; and a computer processor configured to: (a)calculate f_(θ)(x)=R_(d*Pθ(B))(x) for every angle θ, wherein R denotesan autocorrelation operator, P_(θ) denotes a projection operator of atwo dimensional signal into one dimension based on angle θ, and ddenotes a one dimensional differentiation filter applied to a product ofthe projection operator P_(θ) and the blurred image B; (b) set supportparameters s_(θ) to argmin_(x) f_(θ)(x); (c) estimate |{circumflex over(k)}|² denoting a spectral power of the blur kernel based on a givensupport parameter; (d) estimate the blur kernel k using a phaseretrieval algorithm, based on the estimated spectral power of the blurkernel |{circumflex over (k)}|²; (e) updating the support parameterss_(θ) to argmax_(x) (R_(Pθ(k))(x)>a·max(R_(Pθ(k)))), wherein a isconstant number; and (f) repeat the estimating of the spectral power|{circumflex over (k)}|², the estimating of the kernel and the updatingof the support parameters s_(θ) in an expectation maximization (EM)procedure, to yield the blur kernel k.
 9. The system according to claim8, wherein the projection operator P_(θ) is achieved by integrating thetwo dimensional signal along a direction orthogonal to the angle θ. 10.The system according to claim 8, wherein blur kernel k represents theblur operation which transforms I to B due to movements of a capturingdevice upon capturing the blurred image.
 11. The system according toclaim 8, wherein I is a natural image captured by an image capturingdevice.
 12. The system according to claim 8, wherein the angles θ areselected so that each selected angle corresponds with a pixel of thepixel array.
 13. The system according to claim 8, further comprisingusing the retrieved blur kernel k to retrieve the original image. 14.The system according to claim 8, wherein a is between 0 and
 1. 15. Acomputer program product comprising: a non-transitory computer readablestorage medium having computer readable program embodied therewith, thecomputer readable program comprising: computer readable programconfigured to obtain a blurred image B, being a product of a blur kernelk applied to an original image I, wherein B and I are matricesrepresenting pixel image arrays and k is a kernel of a matrix; computerreadable program configured to calculate f_(θ)(x)=R_(d*Pθ(B))(x) forevery angle θ, wherein R denotes an autocorrelation operator, P_(θ)denotes a projection operator of a two dimensional signal into onedimension based on angle θ, and d denotes a one dimensionaldifferentiation filter applied to a product of the projection operatorP_(θ) and the blurred image B; computer readable program configured toset support parameters s_(θ) to argmin_(x) f_(θ)(x); computer readableprogram configured to estimate |{circumflex over (k)}|² denoting aspectral power of the blur kernel based on a given support parameter;computer readable program configured to estimate the blur kernel k usinga phase retrieval algorithm, based on the estimated spectral power ofthe blur kernel |{circumflex over (k)}|²; computer readable programconfigured to updating the support parameters s_(θ) to argmax_(x)(R_(Pθ(k))(x)>a·max(R_(Pθ(k)))), wherein a is constant number; andcomputer readable program configured to repeat the estimating of thespectral power |{circumflex over (k)}|², the estimating of the kerneland the updating of the support parameters s_(θ) in an expectationmaximization (EM) procedure, to yield the blur kernel k.
 16. Thecomputer program product according to claim 15, wherein the projectionoperator P_(θ) is achieved by integrating the two dimensional signalalong a direction orthogonal to the angle θ.
 17. The computer programproduct according to claim 15, wherein blur kernel k represents the bluroperation which transforms I to B due to movements of a capturing deviceupon capturing the blurred image.
 18. The computer program productaccording to claim 15, wherein I is a natural image captured by an imagecapturing device.
 19. The computer program product according to claim15, wherein the angles θ are selected so that each selected anglecorresponds with a pixel of the pixel array.
 20. The computer programproduct according to claim 15, further comprising computer readableprogram configured to use the retrieved blur kernel k to retrieve theoriginal image.